Methods and apparatus to monitor environments

ABSTRACT

Methods and apparatus to monitor environments are disclosed. An example method includes analyzing a plurality of three-dimensional data points having respective depth values representative of distances between a sensor and respective objects of an environment; when a first set of the three-dimensional data points has a first depth value less than a threshold, executing a first type of recognition analysis on a first area of the environment corresponding to the first set of the three-dimensional data points; and when a second set of the three-dimensional data points has a second depth value greater than the threshold, executing a second type of recognition analysis different than the first type of recognition analysis on a second area of the environment corresponding to the second set of the three-dimensional data points.

FIELD OF THE DISCLOSURE

This disclosure relates generally to audience measurement and, moreparticularly, to methods and apparatus to monitor environments.

BACKGROUND

Audience measurement of media (e.g., broadcast television and/or radio,stored audio and/or video content played back from a memory such as adigital video recorder or a digital video disc, a webpage, audio and/orvideo media presented (e.g., streamed) via the Internet, a video game,etc.) often involves collection of media identifying data (e.g.,signature(s), fingerprint(s), code(s), tuned channel identificationinformation, time of exposure information, etc.) and people data (e.g.,user identifiers, demographic data associated with audience members,etc.). The media identifying data and the people data can be combined togenerate, for example, media exposure data indicative of amount(s)and/or type(s) of people that were exposed to specific piece(s) ofmedia.

In some audience measurement systems, the people data is collected bycapturing a series of images of a media exposure environment (e.g., atelevision room, a family room, a living room, a bar, a restaurant,etc.) and analyzing the images to determine, for example, an identity ofone or more persons present in the media exposure environment, an amountof people present in the media exposure environment during one or moretimes and/or periods of time, etc. The collected people data can becorrelated with media detected as being presented in the media exposureenvironment to provide exposure data (e.g., ratings data) for thatmedia.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of an example media exposure environmentincluding an example audience measurement device disclosed herein.

FIG. 1A is another illustration of the example media exposureenvironment of FIG. 1.

FIG. 2 is a block diagram of an example implementation of the exampleaudience measurement device of FIG. 1.

FIG. 2A is a block diagram of an example implementation of the exampleimage capturing device of FIG. 1.

FIG. 3 is a block diagram of an example implementation of the examplepeople counter of FIGS. 1 and/or 2.

FIG. 4 is an illustration of an example face rectangle tracked by theexample people counter of FIGS. 1, 2 and/or 3.

FIGS. 5 and 6 are flowcharts representative of example machine readableinstructions that may be executed to implement the example audiencemeasurement device of FIGS. 1, 2 and/or 3.

FIG. 7 is a block diagram of an example processing system capable ofexecuting the example machine readable instructions of FIGS. 5 and/or 6to implement the example audience measurement device of FIGS. 1, 2and/or 3.

DETAILED DESCRIPTION

There are known applications, systems, and devices (e.g., surveillancesystems, consumer behavior monitors deployed in shopping centers,audience measurement devices, etc.) that benefit from the ability toaccurately count a number of people in a particular space or environmentat a particular time. Such known systems typically capture images of themonitored environment and analyze the images to determine how manypeople are present at certain times.

To count people in a media exposure environment, some known audiencemeasurement systems attempt to recognize objects as humans and/or torecognize an identity of a detected person in image data representativeof the media exposure environment. As used herein, recognition systemsinclude one or more devices, one or more applications, one or morealgorithms, etc. tasked with recognizing and/or detecting one or moreaspects of image data representative of an environment and/or occupantsof the environment. In some known systems, recognition systems includedevices, applications, algorithms, etc. tasked with determining that aportion of image data represents a particular type of object such as,for example, a body part (e.g., an arm, a hand, a leg, a head, a torso,etc.). In some known systems, recognition systems include devices,applications, algorithms, etc. tasked with determining that an object,such as a face, belongs to a certain individual. That is, in some knownsystems, recognition systems include identification systems that, forexample, verify and/or detect an identity of a known person. In someknown examples, recognition systems include devices, applications,algorithms, etc. tasked with identifying a gesture (e.g., a handmovement, a body motion, etc.) performed by a detected person.

In some traditional systems, the audience measurement system maintains atally for each frame of image data to reflect an amount of people in theenvironment at a time corresponding to a respective frame. Recognitionof an object as a human in a frame of image data causes thesetraditional audience measurement devices to increment the tallyassociated with that frame. Some such systems do not attempt to identifythe person, but instead maintain a tally of persons and prompt theaudience to self identify when a change in the count occurs. In exampleswhen face detection is employed, faces of people may be undetected orunrecognized due to, for example, partial visibility, lightingconditions, obscuring of the face due to eating or drinking, or arotation of a head relative to a camera capturing the frames, etc.Additionally, faces of people in the media exposure environment may goundetected or unrecognized in such systems due to field of viewlimitations associated with an image sensor. In other words, the imagesensor tasked with captured images of the media exposure environment maynot have a wide enough field of view to capture certain faces of peoplethat are being exposed to media. Additionally, a non-human object, suchas a picture of a human face hanging on a wall, is sometimes mistakenfor a human face in such known systems, thereby improperly inflating thetally for the corresponding frame.

These and other limitations and/or inaccuracies of known systems canlead to an inaccurate tally of people for individual frames. Aninaccurate tally of people in a frame can negatively affect the accuracyof media exposure data generated using the tally. For example, anaudience measurement system counting the people in a room may also becollecting media identifying information to identify media beingpresented (e.g., aurally and/or visually) in the room. With theidentification of the media and the amount of people in the room at agiven date and time, the audience measurement system can indicate howmany people were exposed to the specific media and/or associate thedemographics of the people with the specific exposure to determineaudience characteristics for the specific media. An audience measuremententity (e.g., The Nielsen Company (US), LLC) can calculate ratings for apiece of media detected in an environment by correlating the piece ofmedia with presence information detected in the environment at timescorresponding to a detection of the piece of media. If face(s) are notdetected or recognized as faces, the exposure data for the identifiedmedia may be under counted (e.g., the media is credited with lessviewers/listeners than had actually been exposed to the media).Alternatively, if false positives are detected, the exposure data forthe identified media may be overstated (e.g., the media is credited withmore viewers/listeners than had actually been exposed to the media).

Recent developments in the capture and processing of three-dimensionaldata have improved recognition systems that previously relied solely ontwo-dimensional image data. That is, systems tasked with, for example,recognizing objects as people, determining an identity of a person,and/or identifying a gesture being made by a person have benefited fromthree-dimensional imaging technology that has improved the accuracy,efficiency, and capabilities of such systems. For example, somethree-dimensional data processing involves generating a skeletalframework representative of a detected person. The skeletal framework isespecially useful in tracking movements of a person and/or identifyinggestures made by the person. Further, capture and processing ofthree-dimensional data can provide a more accurate identification of anobject as a body part (e.g., an arm, a hand, a head, a leg, etc.) thantwo-dimensional data. Further, capture and processing ofthree-dimensional data can provide a more accurate identification of aknown person than two dimensional data. Further, capture and processingof three-dimensional data improves an ability to distinguish actualhuman faces from face-like patterns, such as painting, pictures, fabricsincluding face depictions, etc.

Examples disclosed herein recognize that the advantages provided byusing three-dimensional data to recognize objects as people and/oridentify people are less significant (relative to the use oftwo-dimensional data) when depth values of the three-dimensional dataare beyond a certain distance (which may vary depending on a particularsensor used to capture the three-dimensional data and/or a particularprocessing technique). In other words, examples disclosed hereinrecognize that three-dimensional data and the processing thereof improverecognition of objects to a first degree when the objects are within athreshold distance from the three-dimensional sensor and to a second,less significant degree when the objects are outside the thresholddistance from the three-dimensional sensor. Accordingly, a greaterdistance between the three-dimensional sensor and the analyzed objectleads to less accurate conclusions from analyses generated bythree-dimensional based (3D-based) recognition systems.

For example, a 3D-based recognition analysis tasked with recognizingobjects as human body parts (e.g., arms, legs, heads, hands, etc.) mayhave a first accuracy rating (e.g., a percentage of true positives, apercentage of false positives, a percentage of false negatives, and/or apercentage of true negatives) for recognition of objects located betweenone (1) foot and six (6) feet away from a three-dimensional sensor, asecond accuracy rating for recognition of objects located between six(6) feet and ten (10) feet away from the three-dimensional sensor, and athird accuracy rating for recognition of objects located ten (10) feetor greater from the three-dimensional sensor. In such instances, thefirst accuracy rating is greater than the second accuracy rating, andthe second accuracy rating is greater than the third accuracy rating.Examples disclosed herein recognize that in some instances, the secondand/or third accuracy ratings of the 3D-based recognition analysis maybe less than a fourth accuracy rating associated with a two-dimensionalbased (2D-based) recognition analysis. That is, examples disclosedherein recognize that in certain instances (e.g., when a subject objectis greater than a threshold distance away from a sensor) a 2D-basedrecognition analysis may be better suited (e.g., may provide moreaccurate results) for object recognition than a 3D-based recognitionanalyses. The advantages of 2D-based recognition analyses over the3D-based recognition analyses are especially significant for relativelythin objects (e.g., arms, hands, etc.) at relatively far distances fromthe corresponding sensor.

Further, examples disclosed herein recognize that 2D-based recognitionanalyses are often more costly in terms of computational resources than3D-based recognition analyses. In particular, some 2D-based recognitionanalyses often involve complex processing techniques that consume moreresources (e.g., processing cycles, memory, etc.) than 3D-basedrecognition analyses. For example, while many 2D-based recognitionanalyses involve large amounts of comparisons of large amounts of imagedata (e.g., pixel intensities) to determine which portions of the imagedata (if any) correspond to a person, many 3D-based recognition analysesare able to reference collection(s) of shape libraries to determinewhether depth information corresponds to a person. Thus, the additionaldimension used for the 3D-based analyses enable more efficientrecognition of objects.

Example methods, apparatus, and/or articles of manufacture disclosedherein utilize three-dimensional recognition analysis andtwo-dimensional analysis to more accurately and more efficientlyrecognize objects in an environment than previous recognition systems.In particular, example methods, apparatus, and/or articles ofmanufacture disclosed herein capture three-dimensional data of theenvironment via a three-dimensional sensor and determine whether theenvironment includes an object within a threshold distance from thethree-dimensional sensor. The threshold distance used by examplesdisclosed herein corresponds to, for example, a distance at which theaccuracy rating (e.g., a percentage) of the 3D-based recognitionanalysis falls below a threshold accuracy. For object(s) detected withinthe threshold distance from the three-dimensional sensor, examplemethods, apparatus and/or articles of manufacture disclosed hereinexecute a 3D-based recognition analysis to determine, for example,whether the object(s) are people and/or body part(s) of people. Thus,examples disclosed herein execute the 3D-based recognition analysis onobjects for which the threshold accuracy can be achieved. For each oneof the object(s) determined to correspond to a person via the 3D-basedrecognition analysis, example methods, apparatus, and/or articles ofmanufacture disclosed herein increment a people tally associated withthe environment (e.g., for a particular frame corresponding to thecaptured three-dimensional data). Thus, the people tally generated byexamples disclosed herein for objects located within the thresholddistance have an accuracy aligned with the high accuracy provided by the3D-based recognition analysis.

For portions of the environment having depth information in the capturedthree-dimensional data greater than the threshold distance, examplemethods, apparatus, and/or articles of manufacture disclosed hereinexecute a 2D-based recognition analyses on captured two-dimensional data(e.g., via a two-dimensional image sensor) to determine whether anypeople are present in the environment at distances greater than thethreshold distance associated with the 3D-based recognition analysis.Put another way, examples disclosed herein execute the 2D-basedrecognition analysis for objects located at distances at which theaccuracy of the 3D-based recognition analyses falls below the thresholdaccuracy. As described in greater detail below, example methods,apparatus, and/or articles of manufacture disclosed herein triggerexecution of the 2D-based recognition analysis in response to one ormore example detections and/or events. For example, some examplesdisclosed herein trigger the 2D-based recognition analysis in connectionwith the capture of each frame of captured image data. Additionally oralternatively, some examples disclosed herein trigger the 2D-basedrecognition analysis in response to a predefined time interval elapsing(e.g., according to a schedule). Additionally or alternatively, someexamples disclosed herein trigger the 2D-based recognition analysis inresponse to the 3D-based recognition analysis determining that a personhas moved from within the threshold distance from the sensor to adistance outside the threshold distance. Some such examples disclosedherein use the location of the person that has moved beyond thethreshold distance (as recorded via the 3D-based recognition analysis)to focus the 2D-based recognition analysis on a particular area of theenvironment, thereby avoiding the need to scan the entire environment inconnection with the 2D-based recognition analysis, which is acomputational expensive procedure. In some examples disclosed herein,the 2D-based analysis is only performed in areas beyond the area inwhich 3D-based analyses have an acceptable level of accuracy. Somedisclosed examples use a combination of any two or more of the abovetriggers (e.g., time based and three-dimensional recognitiondetermination) to trigger the 2D-based analysis.

FIG. 1 is an illustration of an example media exposure environment 100including an information presentation device 102, an example imagecapturing device 104, and a meter 106 for collecting audiencemeasurement data. In the illustrated example of FIG. 1, the informationpresentation device 102 is a television and the media exposureenvironment 100 is a room of a household (e.g., a room in a home of apanelist such as the home of a “Nielsen family”) that has beenstatistically selected to develop television ratings data forpopulation(s)/demographic(s) of interest. In the illustrated example,one or more persons of the household have registered with an audiencemeasurement entity (e.g., by agreeing to be a panelist) and haveprovided demographic information to the audience measurement entity toenable associating demographics with viewing activities (e.g., mediaexposure).

In some examples, the audience measurement entity provides the imagecapturing device 104 to the household. In some examples, the imagecapturing device 104 is a component of a media presentation systempurchased by the household such as, for example, a camera of a videogame system 108 (e.g., Microsoft® Kinect®) and/or piece(s) of equipmentassociated with a video game system (e.g., a Kinect® sensor). In suchexamples, the image capturing device 104 may be repurposed and/or datacollected by the image capturing device 104 may be repurposed foraudience measurement.

In the illustrated example of FIG. 1, the image capturing device 104 isplaced above the information presentation device 102 at a position forcapturing image data of the environment 100. In some examples, the imagecapturing device 104 is positioned beneath or to a side of theinformation presentation device 102 (e.g., a television or otherdisplay). In some examples, the image capturing device 104 is integratedwith the video game system 108. For example, the image capturing device104 may collect image data (e.g., three-dimensional data and/ortwo-dimensional data) using one or more sensors for use with the videogame system 108 and/or may also collect such image data for use by themeter 106. In some examples, the image capturing device 104 employs afirst type of image sensor (e.g., a two-dimensional sensor) to obtainimage data of a first type (e.g., two-dimensional data) and collects asecond type of image data (e.g., three-dimensional data) from a secondtype of image sensor (e.g., a three-dimensional sensor). In someexamples, only one type of sensor is provided by the video game system108 and a second sensor is added by the audience measurement system.

In the example of FIG. 1, the meter 106 is a software meter provided forcollecting and/or analyzing the data from the image capturing device 104and other media identification data collected as explained below. In theillustrated example, the meter 106 is installed in the video game system108 (e.g., by being downloaded to the same from a network, by beinginstalled at the time of manufacture, by being installed via a port(e.g., a universal serial bus (USB) from a jump drive provided by theaudience measurement company, by being installed from a storage disc(e.g., an optical disc such as a BluRay disc, Digital Versatile Disc(DVD) or CD (compact Disk), or by some other installation approach).Executing the meter 106 on the panelist's equipment is advantageous inthat it reduces the costs of installation by relieving the audiencemeasurement entity of the need to supply hardware to the monitoredhousehold). In other examples, rather than installing the software meter106 on the panelist's consumer electronics, the meter 106 is a dedicatedaudience measurement unit provided by the audience measurement entity.In such examples, the meter 106 may include its own housing, processor,memory and software to perform the desired audience measurementfunctions. In such examples, the meter 106 is adapted to communicatewith the image capturing device 104 via a wired or wireless connection.In some such examples, the communications are effected via thepanelist's consumer electronics (e.g., via a video game console). Inother example, the image capturing device 104 is dedicated to audiencemeasurement and, thus, no direct interaction with the consumerelectronics owned by the panelist is involved.

Although the example audience measurement system of FIG. 1 isillustrated in a panelist home, the system can be implemented inadditional and/or alternative types of environments such as, forexample, a room in a non-statistically selected household (e.g., anon-panelist site), a theater, a restaurant, a tavern, a retaillocation, an arena, etc. For example, the environment may not beassociated with a panelist of an audience measurement study, but insteadmay simply be an environment associated with a purchased XBOX® and/orKinect® system. In some examples, the example audience measurementsystem of FIG. 1 is implemented, at least in part, in connection withadditional and/or alternative types of media presentation devices suchas, for example, a radio, a computer, a tablet, a cellular telephone,and/or any other communication device able to present media to one ormore individuals.

In the example shown in FIG. 1, first and second persons 109 and 110 arepresent in the environment 100. In the illustrated example, the firstperson 109 is located in a near range set of distances 112 defined by alocation of a three-dimensional sensor of the image capturing device 104in the example of FIG. 1, and a first threshold distance 114 extendingaway from the imaging capturing device 104 along the z-axis shown inFIG. 1. The example environment 100 of FIG. 1 also includes a mid-rangeset of distances 116 defined by a second threshold distance 118extending away from the image capturing device 104 along the z-axis andbeginning at the end of the first threshold distance 114. The exampleenvironment 100 of FIG. 1 also includes a far range set of distances 120that includes distances greater than the second threshold distance 118away from the image capturing device 104.

FIG. 1A illustrates the example media exposure environment 100 of FIG. 1from a different perspective. In particular, the example image capturingdevice 104 captures data across a field of view illustrated in FIG. 1A.As shown in FIG. 1A, a first detection area 122 is defined by the firstthreshold 114 and the field of view to include the near range set ofdistances 112. Further, a second detection area 124 is defined by thefirst threshold 114, the second threshold 118, and the field of view toinclude the mid-range set of distances 116. Further, a third detectionarea 126 is defined by the second threshold 118 and the field of view toinclude the far range set of distances 120. As described in detail belowin connection with FIG. 2, the example meter 106 of FIG. 1 appliesdifferent types of recognition analyses to determine whether objectscorrespond to people depending on, for example, in which one of thedetection areas 122-126 or distance ranges 112, 116 and 120 of theenvironment 100 the objects are located.

FIG. 2 illustrates an example implementation of the meter 106 of FIG. 1.FIG. 2A illustrates an example implementation of the image capturingdevice 104 of FIG. 1. The example meter 106 of FIG. 2 includes anaudience detector 200 to develop audience composition informationregarding, for example, presence of people in the example environment100 of FIG. 1. The example meter 106 of FIG. 2 also includes a mediadetector 202 to collect information regarding, for example, mediapresented in the environment 100 of FIG. 1. An example implementation ofthe media detector 202 is discussed further below. The example imagecapturing device 104 of FIG. 2A includes a three-dimensional sensor 204capable of capturing three-dimensional data representative of theenvironment 100 and a two-dimensional sensor 206 capable of capturingtwo-dimensional data representative of the environment 100. While theexample image capturing device 104 of FIG. 2A includes thethree-dimensional sensor 204 and the two-dimensional sensor 206, theexample meter 106 may additionally or alternatively receivethree-dimensional data and/or two-dimensional data representative of theenvironment 100 from a different source. For example, the imagecapturing device 104 may include the two-dimensional sensor 206 and mayreceive three-dimensional data representative of the environment 100from a three-dimensional sensor implemented by a different component incommunication with the example meter 106 such as, for example, a videogame system (e.g., Microsoft® Kinect®) owned by the panelist. In someexamples, the image capturing device 104 is implemented by a Kinect®sensor which includes both the two-dimensional sensor 206 and thethree-dimensional sensor 204.

The example three-dimensional sensor 204 of FIG. 2A projects an array orgrid of dots (e.g., via one or more electromagnetic radiation beams)onto objects of the environment 100. The dots of the array projected bythe example three-dimensional sensor 204 have respective x-axiscoordinates and y-axis coordinates and/or some derivation thereof. Theexample three-dimensional sensor 204 of FIG. 2A uses feedback receivedin connection with the dot array to calculate depth values associatedwith different dots projected onto the environment 100. Thus, theexample three-dimensional sensor 204 generates a plurality of datapoints. Each such data point has a first component representative of anx-axis position in the environment 100, a second componentrepresentative of a y-axis position in the environment 100, and a thirdcomponent representative of a z-axis position in the environment 100. Asused herein, the x-axis position of an object is referred to as ahorizontal position, the y-axis position of the object is referred to asa vertical position, and the z-axis position of the object is referredto as a depth position relative to the image capturing device 104. Inthe illustrated example, the array projected onto the environment 100 isan infrared array. The example image capturing device 104 of FIG. 2A mayutilize additional or alternative type(s) of three-dimensional sensor(s)to capture three-dimensional data representative of the environment 100such as, for example, image capturing devices employing structuredlighting, time-of-flight of light, and/or stereo cameras.

The example three-dimensional sensor 204 of FIG. 2A is implemented by alaser that projects a plurality grid points onto the environment 100.The example two-dimensional sensor 206 of FIG. 2A is implemented by asensor and/or camera that captures two-dimensional image datarepresentative of the environment 100. In some examples, thetwo-dimensional sensor 206 includes an infrared imager, a complimentarymetal-oxide semiconductor (CMOS) camera, and/or a charge coupled device(CCD) camera.

In some examples, the sensors three-dimensional sensor 204 and/or thetwo-dimensional sensor 206 only capture data when the informationpresentation device 102 is in an “on” state and/or when the mediadetector 202 determines that media is being presented in the environment100 of FIG. 1.

The example audience detector 200 of FIG. 2 includes a people analyzer208, a time stamper 210 and a memory 212. In the illustrated example ofFIG. 2, the data generated by the example three-dimensional sensor 204and/or the example two-dimensional sensor 206 is conveyed to the examplepeople analyzer 208. The example people analyzer 208 of FIG. 2calculates a people count corresponding to different time periods (e.g.,one minute intervals, thirty second intervals, etc.) for the exampleenvironment 100 of FIG. 1. The manner in which the example peopleanalyzer 208 of FIG. 2 generates the people counts is described indetail below in connection with FIGS. 3-6.

The example people analyzer 208 of FIG. 2 outputs, for example,calculated people counts or tallies to the example time stamper 210. Thetime stamper 210 of the illustrated example includes a clock and acalendar. The example time stamper 210 associates a time period (e.g.,1:00 a.m. Central Standard Time (CST) to 1:01 a.m. CST) and date (e.g.,Jan. 1, 2012) with each calculated people count by, for example,appending the period of time and date information to an end of thepeople data. In some examples, the time stamper 210 applies a singletime and date rather than a period of time. A data package (e.g., thepeople count, the time stamp, the image data, etc.) is stored in thememory 212.

The example memory 212 of FIG. 2 may include a volatile memory (e.g.,Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random AccessMemory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM, etc.) and/ora non-volatile memory (e.g., flash memory). The example memory 212 ofFIG. 2 may also include one or more mass storage devices such as, forexample, hard drive disk(s), compact disk drive(s), digital versatiledisk drive(s), etc. When the example meter 106 is integrated into, forexample, the video game system 108 of FIG. 1, the meter 106 may utilizememory of the video game system 108 to store information such as, forexample, the people counts, the image data, etc.

The example time stamper 210 of FIG. 2 also receives data from theexample media detector 202. The example media detector 202 of FIG. 2detects presentation(s) of media in the media exposure environment 100and/or collects identification information associated with the detectedpresentation(s). For example, the media detector 202, which may be inwired and/or wireless communication with the information presentationdevice 102 (e.g., a television), the video game system 108, an STBassociated with the information presentation device 102, and/or anyother component of FIG. 1, can identify a presentation time and/or asource of a presentation. The presentation time and the sourceidentification data may be utilized to identify the program by, forexample, cross-referencing a program guide configured, for example, as alook up table. In such instances, the source identification data is, forexample, the identity of a channel (e.g., obtained by monitoring a tunerof an STB or a digital selection made via a remote control signal)currently being presented on the information presentation device 102.

Additionally or alternatively, the example media detector 202 canidentify the presentation by detecting codes and/or watermarks embeddedwith or otherwise conveyed (e.g., broadcast) with media being presentedvia an STB and/or the information presentation device 102. As usedherein, a code is an identifier that is transmitted with the media forthe purpose of identifying and/or for tuning to (e.g., via a packetidentifier (PID) header and/or other data used to tune or select packetsin a multiplexed stream of packets) the corresponding media. Codes maybe carried in the audio, in the video, in metadata, in a verticalblanking interval, in a program guide, in content data, or in any otherportion of the media and/or the signal carrying the media. In theillustrated example, the media detector 202 extracts the code(s) fromthe media. In other examples, the media detector may collect samples ofthe media and export the samples to a remote site for detection of thecode(s).

Additionally or alternatively, the media detector 202 can collect asignature representative of a portion of the media. As used herein, asignature is a representation of some characteristic of the signalcarrying or representing one or more aspects of the media (e.g., afrequency spectrum of an audio signal). Signatures may be thought of asfingerprints of the media. Collected signature(s) can be comparedagainst a collection of reference signatures of known media (e.g.,content and/or advertisements) to identify tuned media. In someexamples, the signature(s) are generated by the media detector 202.Additionally or alternatively, the media detector 202 collects samplesof the media and exports the samples to a remote site for generation ofthe signature(s). In the example of FIG. 2, irrespective of the mannerin which the media of the presentation is identified (e.g., based ontuning data, metadata, codes, watermarks, and/or signatures), the mediaidentification information is time stamped by the time stamper 210 andstored in the memory 212.

In the illustrated example of FIG. 2, an output device 214 periodicallyand/or aperiodically exports the audience identification informationand/or the media identification information from the memory 212 to adata collection facility 216 via a network (e.g., a local-area network,a wide-area network, a metropolitan-area network, the Internet, adigital subscriber line (DSL) network, a cable network, a power linenetwork, a wireless communication network, a wireless mobile phonenetwork, a Wi-Fi network, etc.). In some examples, the example meter 106utilizes the communication capabilities (e.g., network connections) ofthe video game system 108 to convey information to, for example, thedata collection facility 216. In the illustrated example of FIG. 2, thedata collection facility 216 is managed and/or owned by an audiencemeasurement entity (e.g., The Nielsen Company (US), LLC). The audiencemeasurement entity associated with the example data collection facility216 of FIG. 2 utilizes the people tallies generated by the peopleanalyzer 208 in conjunction with the media identifying data collected bythe media detector 202 to generate exposure information. The informationfrom many panelist locations may be collected and analyzed to generateratings representative of media exposure by one or more populations ofinterest.

While example manners of implementing the image capturing device 104 andthe meter 106 of FIG. 1 have been illustrated in FIGS. 2 and 2A, one ormore of the elements, processes and/or devices illustrated in FIGS. 2and/or 2A may be combined, divided, re-arranged, omitted, eliminatedand/or implemented in any other way. Further, the example audiencedetector 200, the example media detector 202, the examplethree-dimensional sensor 204, the example two-dimensional sensor 206,the example people analyzer 208, the example time stamper 210, theexample output device 214, and/or, more generally, the example meter 106of FIG. 2 and/or the example image capturing device 104 may beimplemented by hardware, software, firmware and/or any combination ofhardware, software and/or firmware. Thus, for example, any of theexample audience detector 200, the example media detector 202, theexample three-dimensional sensor 204, the example two-dimensional sensor206, the example people analyzer 208, the example time stamper 210, theexample output device 214, and/or, more generally, the example meter 106of FIG. 2 and/or the example image capturing device 104 of FIG. 2A couldbe implemented by one or more circuit(s), programmable processor(s),application specific integrated circuit(s) (ASIC(s)), programmable logicdevice(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)),etc. When any of the system or apparatus claims of this patent are readto cover a purely software and/or firmware implementation, at least oneof the example audience detector 200, the example media detector 202,the example three-dimensional sensor 204, the example two-dimensionalsensor 206, the example people analyzer 208, the example time stamper210, the example output device 214, and/or, more generally, the examplemeter 106 of FIG. 2 and/or the example image capturing device 104 ofFIG. 2A are hereby expressly defined to include a tangible computerreadable storage medium such as a storage device or storage disc storingthe software and/or firmware. Further still, the example meter 106 ofFIG. 2 and/or the example image capturing device 104 may include one ormore elements, processes and/or devices in addition to, or instead of,those illustrated in FIGS. 2 and/or 2A, and/or may include more than oneof any or all of the illustrated elements, processes and devices.

FIG. 3 illustrates an example implementation of the example peopleanalyzer 208 of FIG. 2. The example people analyzer 208 of FIG. 2generates a people count or tally representative of a number of peoplein the media exposure environment 100 of FIG. 1 for frame(s) of capturedimage data. The rate at which the example people analyzer 208 of FIG. 3generates people counts is configurable. In the illustrated example ofFIG. 3, the example people analyzer 208 instructs the examplethree-dimensional sensor 204 of FIG. 2A to capture three-dimensionaldata representative of the environment 100 every five seconds. However,the example people analyzer 208 can capture and/or analyze data at anysuitable rate. As described in detail below, the example people analyzer208 of FIG. 3 utilizes a three-dimensional data analyzer 300 and atwo-dimensional data analyzer 302 to detect people in the environment100. The example people analyzer 208 of FIG. 3 also includes a bodytracker 304 to store information generated by the data analyzers 300 and302 and information related thereto. Further, as described in detailbelow, the example body tracker 304 of FIG. 3 enables thetwo-dimensional data analyzer 302 to focus a corresponding analysis onareas of interest in the environment 100, rather than analyzing theentirety of the environment 100.

The example three-dimensional data analyzer 300 of FIG. 3 receivesframes of three-dimensional data and executes one or more 3D-basedrecognition analyses on objects appearing in the environment 100 withinone or more threshold distances from the three-dimensional sensor 204.As described above, the received three-dimensional data includes datapoints each having a horizontal component, a vertical component, and adepth component. The example three-dimensional data analyzer 300includes a depth filter 306 to analyze the depth component of the datapoints. The example depth filter 306 of FIG. 3 determines whetherobjects of the environment 100 corresponding to the data points fallwithin the first detection area 122 (e.g., at one of the near range setof distances 112), the second detection area 124 (e.g., at one of themid-range set of distances 116), or the third detection area 126 (e.g.,at one of the far range set of distances 120) of FIG. 1A. In particular,the example depth filter 306 determines whether the depth components ofthe respective data points are (1) less than the first thresholddistance 114, (2) greater than the first threshold distance 114 but lessthan the second threshold distance 118, or (3) greater than the secondthreshold distance 118.

In the illustrated example, the depth filter 306 of FIG. 3 designatesthe data points corresponding to objects of the environment 100 locatedin the first detection area 122 for further 3D-based recognitionanalysis via the example three-dimensional data analyzer 300. Althoughthe example depth filter 306 of FIG. 3 designates the data points foundin the first detection area 122 for 3D-based recognition analysis, suchdata points may be additionally or alternatively be designated for2D-based recognition analysis. Further, in some examples, the firstdetection area 122 includes a subset of distances from the imagecapturing device 104 for which three-dimensional data processing isundesirable due to, for example, limitations on capture ofthree-dimensional data and/or processing thereof. Such a subset ofdistances may include, for example, distances within one (1) foot orwithin one and a half (1.5) feet from the image capturing device 104. Insuch instances, the example depth filter 306 designates the data pointsin the subset of distances for 2D-based recognition. Alternatively, theexample depth filter 306 of FIG. 3 may designate any data point detectedwithin the first detection area 122 for further 3D-based recognitionanalysis.

For data points located in the second detection area 124, the exampledepth filter 306 of FIG. 3 designates the data for further analysis viathe example three-dimensional analyzer 300 and for analysis via thetwo-dimensional data analyzer 302. That is, in the illustrated exampleof FIG. 3, the data points falling in the mid-range set of distances 116are subject to a 2D-based recognition analysis in addition to a 3D-basedrecognition analysis.

The example depth filter 306 of FIG. 3 filters the data pointscorresponding to objects of the environment 100 located in the thirddetection area 126 (e.g., at one of the far range set of distances 120)from further analysis via the example three-dimensional data analyzer300. In the illustrated example of FIG. 3, the accuracy of thethree-dimensional data analyzer 300 is below a threshold (e.g., anaccuracy percentage) for objects located in the third detection area126. Thus, the data points associated with third detection area 126 arenot further analyzed by the example three-dimensional data analyzer 300.As a result, people counts having an accuracy less than a threshold areavoided. That is, the example three-dimensional data analyzer 300 onlygenerates people counts greater than or equal to the relatively highaccuracy rating corresponding to 3D-based recognition of near objects.Alternatively, the example three-dimensional data analyzer 300 mayanalyze the data points corresponding to the far range 120 and theexample depth filter 306 can prevent the results of such analysis fromcontributing or affecting the people counts generated by the examplethree-dimensional data analyzer 300. For example, the depth filter 306may discard analysis results for data points found in the far range 120by preventing a write to memory associated with such analysis results orby writing such results to a separate variable (e.g., an untrustworthytally) to be used, for example, for diagnostic purposes.

In some examples, the three-dimensional sensor 204 supports additionalor alternative ranges and/or the example depth filter 306 designates thedata points for further analysis in additional or alternative manners.For example, the environment 100 may only include a near range (e.g., acombination of the first and second detection areas 122, 124 in FIG. 1A)and a far range (e.g., the third detection area 126 of FIG. 1A). In suchinstances, the example depth filter 306 of FIG. 3 only designates datapoints corresponding to objects located in the near range for furtheranalysis via the example three-dimensional data analyzer 300.

The example body recognizer 308 of FIG. 3 receives the data pointsdetermined to be suitable for further processing via thethree-dimensional data analyzer 300 by the example depth filter 306. Theexample body recognizer 308 of FIG. 3 implements one or more 3D-basedrecognition analyses capable of identifying a set of data points ascorresponding to a body and/or a body part of a person. For example, thebody recognizer 308 of FIG. 3 implements a first 3D-based recognitionanalysis to determine whether a set of three-dimensional data pointscorresponds to a human head, a second 3D-based recognition analysis todetermine whether a set of three-dimensional data points corresponds toa human arm, a third 3D-based recognition analysis to determine whethera set of three-dimensional data points corresponds to a human torso,etc. The recognition analys(es) implemented by the example bodyrecognizer 308 of FIG. 3 are based on, for example, three-dimensionalshapes known to correspond to the corresponding body part(s) and/or atraining algorithm that improves over time based on learned information.

When the example body recognizer 308 of FIG. 3 recognizes a set of datapoints as corresponding to a person, the example body recognizer 308conveys an indication of such a recognition to a body counter 316 of theexample body tracker 304. The example body counter 316 of FIG. 3maintains counts for respective frames of image data representative ofthe environment 100 of FIG. 1. The counts generated and maintained bythe example body counter 316 are stored in association with thecorresponding frames in a frame database 318 of the example body tracker304. In some examples, the frames of image data are also stored in theexample frame database for purposes of, for example, record keepingand/or subsequent verification procedures.

In the illustrated example of FIG. 3, the body recognizer 308 alsoconveys a coordinate of the detected body to a crossover detector 320 ofthe example three-dimensional data analyzer 300. The coordinate of thedetected body is indicative of a location in the environment 100 of thebody and/or body part detected by the body recognizer 308. FIG. 4illustrates an example person 400 detected by the example bodyrecognizer 308 and an example coordinate 402 conveyed to the examplecrossover detector 320. The example of FIG. 4 corresponds to an instanceof the body recognizer 308 detecting a head of the person 400 and, thus,the coordinate 402 corresponds to a center of the detected head.However, when the example body recognizer 308 of FIG. 3 recognizes adifferent body part, the coordinate conveyed to the crossover detector320 is the coordinate that corresponds to that body part. The examplecoordinate 402 of FIG. 4 includes a horizontal component (x-axis) and avertical component (y-axis) corresponding to a two-dimensional locationof the detected person 400 in the environment 100.

The example crossover detector 320 maintains records of the coordinatesreceived in connection with chronologically captured frames and uses thesame to detect a person travelling from the first detection area 122(e.g., at one of the near range set of distances 112) to the seconddetection area 124 (e.g., at one of the mid-range set of distances 116)and/or from the second detection area 124 to the third detection area126 (e.g., at one of the far range set of distances 120) of theenvironment 100. That is, the example crossover detector 320 identifiesinstances of a person crossing over from a first range designated for a3D-based recognition analysis to a second range designated for a2D-based recognition analysis (in addition to or in lieu of a 3D-basedrecognition analysis). For example, the crossover detector 320 of FIG. 3may determine that a change in depth value of a three-dimensional datapoint at the provided coordinate from a first frame to a second framecrosses over one of the threshold distances 114 and 118 of FIG. 1. Asdescribed in detail below in connection with the two-dimensional dataanalyzer 302, the example crossover detector 320 of FIG. 3 provides thelocation (e.g., x-y coordinate) of a person crossing over to a fartherrange such that a 2D-based recognition analysis can be focused on thecorresponding area of the environment 100, thereby reducing the amountof processing of two-dimensional data by avoiding the need for the2D-based recognition analysis to scan the entirety of the environment100.

When the example body recognizer 308 of FIG. 3 determines that a body ispresent in the environment 100, the type of detected body part and thelocation thereof is conveyed to the skeletal frame generator 310. Theexample skeletal frame generator 310 of FIG. 3 analyzes thethree-dimensional data points to generate a representation of a currentshape of the detected body. For example, the skeletal frameworkgenerator 310 of FIG. 3 generates a plurality of centerline pointscorresponding to detected body parts and the relation of the detectedbody parts to other body parts of the detected person. In some examples,the skeletal framework generator 310 determines a posture of the person.For example, the skeletal framework generator 310 may determine that thedetected body is standing, sitting, lying down, etc.

Further, when the example body recognizer 308 of FIG. 3 determines thata body is present in the environment 100, the body recognizer 308 and/orthe skeletal framework generator 310 conveys information to the facerecognizer 312. In the illustrated example, the face recognizer 312attempts to identify the detected body as belonging to one of aplurality of known people, such as a family of the household, frequentvisitors of the household, etc. The example face recognizer 312 employsany suitable facial recognition technique and/or procedure based onthree-dimensional data points. When the example face recognizer 312identifies the detected body as corresponding to a known person, theexample face recognizer 312 conveys the detected identity of the person(e.g., an identifier) to the example body tracker 304. In theillustrated example of FIG. 3, the identity of the detected body isstored in the frame database 318 in association with the appropriateframe of data and/or timestamp.

Further, when the example body recognizer 308 of FIG. 3 determines thata body is present in the environment 100, the type of detected bodypart, the location of the detected body part, and/or data generated bythe example skeletal framework generator 310 is conveyed to the gesturerecognizer 314. The example gesture recognizer 314 uses data associatedwith a single frame and/or data related to several consecutive frames todetermine whether the detected body is making a known gesture (e.g., agesture corresponding to one or more of a collection of predefinedgestures). For example, the detected person may be in the act of makinga gesture towards the information presentation device 102 in connectionwith the video game system 108 of FIG. 1. Such gestures include, forexample, hand motions, body motions, finger configuration, etc. In suchinstances, the example gesture recognizer 314 compares thethree-dimensional data points to data known to correspond to certaingestures. In some examples, information generated by the example gesturerecognizer 314 is conveyed to the example body tracker 304 for storagein, for example, the frame database 318 in association with a timestamp.

Thus, the example three-dimensional data analyzer 300 of FIG. 3 performsone or more 3D-based recognition analyses on data points determined tocorrespond to objects located within one or more threshold distancesfrom the example three-dimensional sensor 204 of FIG. 2A. In theillustrated example of FIG. 3, the three-dimensional data analyzer 300performs the 3D-based recognition analys(es) for ones of the data pointscaptured by the three-dimensional sensor 204 having a depth valuefalling in the first or second detection areas 122 and 124 of FIG. 1A(e.g., in the near range 112 or the mid-range 116).

For ones of the data points having a depth value falling in the seconddetection area 124 or the third detection area of FIG. 1, the exampletwo-dimensional data analyzer 302 of FIG. 3 initiates one or more2D-based recognition analyses. Further, as described above, thetwo-dimensional data analyzer 302 may initiate a 2D-based recognitionanalysis for data points located at a subset of distances in the nearrange of distances 112 (e.g., data points within one (1) foot of theimage capturing device 104, for which one or more 3D-based analysis areimpractical (e.g., depending on the capabilities of thethree-dimensional sensor 204).

In the illustrated example of FIG. 3, the two-dimensional data analyzer302 includes a body recognizer 322 to implement one or more 2D-basedrecognition analyses on two-dimensional data captured by, for example,the two-dimensional sensor 206 of FIG. 2A. Like the example bodyrecognizer 308 of the three-dimensional data analyzer 300 of FIG. 3, theexample body recognizer 322 of the two-dimensional data analyzer 302attempts to recognize objects present in the environment 100 as people.For example, the body recognizer 322 of FIG. 3 utilizes a plurality ofcomparison techniques and/or algorithms to detect shape(s) correspondingto body part(s) such as, for example, an arm, a leg, a hand, a head,etc. As described above, the example body recognizer 308 of thethree-dimensional data analyzer 300 of FIG. 3 is more accurate andefficient than the example body recognizer 322 of the two-dimensionaldata analyzer 302 of FIG. 3 when the body recognizer 308 of thethree-dimensional data analyzer 300 is analyzing data points having adepth value less than a threshold distance. However, the example bodyrecognizer 322 of the two-dimensional data analyzer 302 is more accuratethan the body recognizer 308 of the three-dimensional data analyzer 300when the body recognizer 308 of the three-dimensional data analyzer 300is analyzing data points having a depth value greater than the thresholddistance. Accordingly, the example people analyzer 208 of FIG. 3utilizes the more accurate of the two body recognizers 308 and 322depending on, for example, a depth value of an object to be analyzed.

When the example body recognizer 322 of FIG. 3 detects a body of aperson in a particular frame of two-dimensional data, an indication ofthe recognition is conveyed to the example body counter 316 and/or theexample frame database 318. In the illustrated example of FIG. 3, theexample body counter 316 combines the body detections provided by thebody recognizer 308 of the three-dimensional data analyzer 300 and thebody detections provided by the body recognizer 322 of thetwo-dimensional data analyzer 302 corresponding to a common frame orframes to generate body counts for the respective frames. In someexamples, the body counter 316 also executes one or more filters and/orcheck functions to avoid counting the same person more than once.Further, the example body recognizer 322 calculates and/or stores alocation of the detected person in, for example, the frame database 318in connection with a timestamp. In some examples, the location of thedetected sent by the example body recognizer 322 is an x-y coordinatesimilar to the example coordinate 402 of FIG. 4.

In addition to or in lieu of the example body recognizer 322, theexample two-dimensional data analyzer 302 of FIG. 3 includes a facerecognizer 324 to obtain an identity of a person by analyzingtwo-dimensional data captured by, for example, the two-dimensionalsensor 206. The example face recognizer 324 of FIG. 3 utilizes anysuitable technique(s) and/or algorithm(s) to perform one or moreidentity recognition analyses. The example two-dimensional data analyzer302 of FIG. 3 can implement additional or alternative recognitionanalys(es) that are based on two-dimensional data.

In some examples, the 2D-based recognition analys(es) performed by theexample two-dimensional data analyzer 302 are initiated for each frameof data. However, in the illustrated example of FIG. 3, thetwo-dimensional data analyzer 302 includes an analysis trigger 326 todetermine whether one or more of the 2D-based recognition analyses(e.g., the analysis performed by the body recognizer 322 and/or theanalysis performed by the face recognizer 324) are to be performed on acertain frame of data and/or what type of 2D-based recognition analysisis to be performed. In particular, the example analysis trigger 326 ofFIG. 3 triggers 2D-based recognition analyses in different circumstancesand/or in response to different events. For example, the analysistrigger 326 of FIG. 3 triggers a 2D-based recognition analysis inresponse to arrival of a scheduled time (e.g., elapsing of a predefinedtime interval, occurrences of an absolute time, such as five minutesafter an event, etc.). Additionally or alternatively, the exampleanalysis trigger 326 of FIG. 3 may trigger a 2D-based recognitionanalysis in response to the crossover detector 320 determining that aperson has travelled away from the three-dimensional sensor 204 into,for example, the mid-range 116 and/or the far range 120 of FIG. 1.Additionally or alternatively, the example analysis trigger 326 of FIG.3 may trigger a 2D-based recognition analysis for a first frame inresponse to a 2D-based recognition of a second frame prior to the firstframe resulting in detection of person. The example analysis trigger 326of FIG. 3 may employ additional or alternative events, schedules, etc.for triggering one or more 2D-based recognition analysis.

Moreover, the example analysis trigger 326 of FIG. 3 initiates differenttypes of 2D-based recognition analyses based on one or more factorsand/or conditions. For example, the analysis trigger 326 of FIG. 3sometimes triggers a full scan of the captured two-dimensional data(e.g., via the two-dimensional sensor 206). Additionally oralternatively, the example analysis trigger 326 of FIG. 3 triggers ascan of the captured two-dimensional data (e.g., via the two-dimensionalsensor 206) corresponding to area(s) of the environment 100 having depthvalues greater than a threshold distance. Such area(s) of theenvironment 100 correspond to, for example, the far range 120 and/or themid-range 116. Additionally or alternatively, the example analysistrigger 326 of FIG. 3 triggers a focused analysis that targets a subsetof the captured two-dimensional data corresponding to a previouslydetected person.

The type of 2D-based recognition analysis triggered by the exampleanalysis trigger 326 depends on, for example, which type of event orcircumstance triggered the 2D-based recognition. For example, when the2D-based recognition analysis is triggered in response to expiration ofa first period of time or interval of a schedule elapsing, the exampleanalysis trigger 326 of FIG. 3 selects a full scan of thetwo-dimensional data (e.g., via the face recognizer 324, via the bodyrecognizer 322, etc.) that analyzes the entire environment 100 capturedby the two-dimensional sensor 206. Additionally, when the 2D-basedrecognition is triggered in response to a expiration of a second periodof time or interval of the schedule elapsing, the example analysistrigger 326 of FIG. 3 selects a scan of areas of the environment 100having a depth value (e.g., according to the three-dimensional datapoints) greater than a threshold distance. In some examples, when the2D-based recognition analysis is triggered in response to a detection ofa person crossing over into the third detection area 126 and/or thesecond detection area 124, the example analysis trigger 326 of FIG. 3selects a targeted scan (e.g., via the face recognizer 324, via the bodyrecognizer 322, etc.) of the two-dimensional data focused on aparticular area of the environment 100 corresponding to the detectedperson. In some examples, when the 2D-based recognition analysis istriggered for a first frame in response to a detection of a person in asecond, prior frame via a 2D-based recognition analysis, the exampleanalysis trigger 326 of FIG. 3 selects a targeted scan of thetwo-dimensional data focused on a particular area of the environment 100corresponding to the previously detected person. The example analysistrigger 326 of FIG. 3 may employ and/or select additional or alternativetypes of 2D-based recognition analyses.

To implement the focused type of 2D-based recognition analysis that islimited to a particular area of the environment 100, the exampletwo-dimensional data analyzer 302 of FIG. 3 includes an analysis limiter328. The example analysis limiter 328 of FIG. 3 obtains locationinformation associated with previously detected people to focus the2D-based recognition analysis executed by the two-dimensional dataanalyzer 302 to certain area(s) of the environment 100. For example, theanalysis limiter 328 of FIG. 3 obtains a coordinate provided by theexample crossover detector 320 that corresponds to a person crossingover into the second detection area 124 and/or the third detection area126 of FIG. 1A. In some instances, the coordinate used by the exampleanalysis limiter 328 corresponds to the example coordinate 402 of FIG.4. As described above, a person recognized while located in the firstdetection area 122 (e.g., by the example body recognizer 308 of thethree-dimensional data analyzer 300) may be determined to have travelledoutside the first detection area 122 (e.g., into the mid-range set ofdistances 116 and/or the far range set of distances 120) and, thus, a2D-based recognition analysis is selected by the people analyzer 208 toanalyze such data (e.g., because the 2D-based recognition analysis ismore accurate in detecting the person in subsequent frames than 3D-basedrecognition analysis for object located at certain depths of theenvironment). In such instances, the example analysis limiter 328focuses the 2D-based recognition analysis in a certain area (e.g., thearea where the person is expected to be) to avoid having to execute the2D-based recognition analysis on the entire environment 100, which iscomputationally expensive. Alternatively, when the triggered 2D-basedrecognition analysis corresponds to a person detected via a previousexecution of the 2D-based face recognizer 324 in a prior frame, thelocation obtained by the example analysis limiter 328 of FIG. 3 is acoordinate generated by the example body recognizer 322. That is, whenthe example face recognizer 324 of FIG. 3 (and/or body recognizer 322 ofFIG. 3) detects a person in a first frame, the face recognizer 324records the location of the person (e.g., an x-y coordinate such as theexample coordinate 402 of FIG. 4). In such instances, the exampleanalysis trigger 326 may trigger the face recognizer 324 to execute a2D-based recognition analysis for a second, subsequent frame. If so, theexample analysis limiter 328 of FIG. 3 uses the location recorded inconnection with the first frame to focus the 2D-based recognitionanalysis on a corresponding part of the second frame. The exampleanalysis limiter 328 of FIG. 3 focuses recognition analyses by, forexample, defining boundaries around the obtained location (e.g.,coordinate) and using the boundaries to limit the 2D-based recognitionanalysis within the boundaries. Put another way, the example analysislimiter 328 of FIG. 3 selects a portion or subset of the two-dimensionaldata captured by the two-dimensional sensor 206 for the 2D-basedrecognition analysis performed by, for example, the face recognizer 324.

The example analysis limiter 328 generates different types of boundariesthat form different shapes around a focal point (e.g., the obtainedcoordinate in which analysis is to be performed). The shape created bythe boundaries of the example analysis limiter 328 of FIG. 3 depends on,for example, which body part was detected by the body recognizer 322 inthe previous frame. For example, when the body recognizer 322 hasdetermined that an object in the environment 100 is likely a human head(and, thus, the focal coordinate is the center of the detected head),the example analysis limiter 328 of FIG. 3 defines boundaries that forma rectangle surrounding the obtained location point. Alternatively, whenthe body recognizer 322 has determined that an object in the environment100 is likely a torso (and, thus, the focal coordinate is the center ofthe torso), the example analysis limiter 328 of FIG. 3 definesboundaries that form an oval surrounding the focal location point. Theexample analysis limiter 328 of FIG. 3 may use additional or alternativetypes of boundaries and/or shapes to focus the 2D-based recognitionanalysis performed by the example two-dimensional data analyzer 302 ofFIG. 3 on specific area(s).

While the example people analyzer 208 of FIGS. 2 and/or 3 is describedin the context of an audience measurement device system and thegeneration of exposure data for media, the example methods, articles ofmanufacture, and apparatus disclosed herein can be applied to additionalor alternative environments, contexts, systems, measurements,applications, programs, problems, etc. That is, the example methods,articles of manufacture, and apparatus disclosed herein can be used inany application wherein detecting to determine how many people arelocated in a space or location would be beneficial (e.g., in securityapplications).

While example manners of implementing the people analyzer 208 of FIG. 2have been illustrated in FIG. 3, one or more of the elements, processesand/or devices illustrated in FIG. 3 may be combined, divided,re-arranged, omitted, eliminated and/or implemented in any other way.Further, the example three-dimensional analyzer 300, the exampletwo-dimensional analyzer 302, the example body tracker 304, the exampledepth filter 306, the example body recognizer 308, the example skeletalframe generator 310, the example face recognizer 312, the examplegesture recognizer 314, the example body counter 316, the examplecrossover detector 320, the example body recognizer 322, the exampleface recognizer 324, the example analysis trigger 326, the exampleanalysis limiter 328, and/or, more generally, the example peopleanalyzer 208 of FIG. 3 may be implemented by hardware, software,firmware and/or any combination of hardware, software and/or firmware.Thus, for example, any of the example three-dimensional analyzer 300,the example two-dimensional analyzer 302, the example body tracker 304,the example depth filter 306, the example body recognizer 308, theexample skeletal frame generator 310, the example face recognizer 312,the example gesture recognizer 314, the example body counter 316, theexample crossover detector 320, the example body recognizer 322, theexample face recognizer 324, the example analysis trigger 326, theexample analysis limiter 328, and/or, more generally, the example peopleanalyzer 208 of FIG. 3 could be implemented by one or more circuit(s),programmable processor(s), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)), etc. When any of the system orapparatus claims of this patent are read to cover a purely softwareand/or firmware implementation, at least one of the examplethree-dimensional analyzer 300, the example two-dimensional analyzer302, the example body tracker 304, the example depth filter 306, theexample body recognizer 308, the example skeletal frame generator 310,the example face recognizer 312, the example gesture recognizer 314, theexample body counter 316, the example crossover detector 320, theexample body recognizer 322, the example face recognizer 324, theexample analysis trigger 326, the example analysis limiter 328, and/or,more generally, the example people analyzer 208 of FIG. 3 are herebyexpressly defined to include a tangible computer readable storage mediumsuch as a storage device or storage disc storing the software and/orfirmware. Further still, the example people analyzer 208 may include oneor more elements, processes and/or devices in addition to, or insteadof, those illustrated in FIG. 3, and/or may include more than one of anyor all of the illustrated elements, processes and devices.

FIGS. 5 and 6 are flowcharts representative of example machine readableinstructions for implementing the example people analyzer 208 of FIGS. 2and/or 3. In the example flowcharts of FIGS. 5 and 6, the machinereadable instructions comprise program(s) for execution by a processorsuch as the processor 712 shown in the example processing platform 700discussed below in connection with FIG. 7. The program(s) may beembodied in software stored on a tangible computer readable storagemedium such as a CD-ROM, a floppy disk, a hard drive, a digitalversatile disk (DVD), a Blu-ray disk, or a memory associated with theprocessor 712, but the entire program and/or parts thereof couldalternatively be executed by a device other than the processor 712and/or embodied in firmware or dedicated hardware. Further, although theexample program(s) is described with reference to the flowchartsillustrated in FIGS. 5 and 6, many other methods of implementing theexample people analyzer 208 may alternatively be used. For example, theorder of execution of the blocks may be changed, and/or some of theblocks described may be changed, eliminated, or combined.

As mentioned above, the example processes of FIGS. 5 and 6 may beimplemented using coded instructions (e.g., computer readableinstructions) stored on a tangible computer readable storage medium suchas a hard disk drive, a flash memory, a read-only memory (ROM), acompact disk (CD), a digital versatile disk (DVD), a cache, arandom-access memory (RAM) and/or any other storage media in whichinformation is stored for any duration (e.g., for extended time periods,permanently, brief instances, for temporarily buffering, and/or forcaching of the information). As used herein, the term tangible computerreadable storage medium is expressly defined to include any type ofcomputer readable storage and to exclude propagating signals.Additionally or alternatively, the example processes of FIGS. 5 and 6may be implemented using coded instructions (e.g., computer readableinstructions) stored on a non-transitory computer readable medium suchas a hard disk drive, a flash memory, a read-only memory, a compactdisk, a digital versatile disk, a cache, a random-access memory and/orany other storage media in which information is stored for any duration(e.g., for extended time periods, permanently, brief instances, fortemporarily buffering, and/or for caching of the information). As usedherein, the term non-transitory computer readable medium is expresslydefined to include any type of computer readable medium and to excludepropagating signals. As used herein, when the phrase “at least” is usedas the transition term in a preamble of a claim, it is open-ended in thesame manner as the term “comprising” is open ended. Thus, a claim using“at least” as the transition term in its preamble may include elementsin addition to those expressly recited in the claim.

FIG. 5 begins with an initiation of the example people analyzer 208 ofFIG. 3 (block 500). The initiation of the people analyzer 208 coincideswith, for example, activation of the information presentation device 102and/or the video game system 108 of FIG. 1. To begin analyzing theenvironment 100 of FIG. 1, the three-dimensional sensor 204 of FIG. 2Acaptures three-dimensional data representative of the environment 100and the objects present in the environment 100 at a first time (block502). The example three-dimensional sensor captures a plurality ofthree-dimensional data points having a horizontal component, a verticalcomponent, and a depth value component. The example depth filter 306 ofFIG. 3 analyzes the data points to determine whether the respectivedepth values are within one or more threshold distances from thethree-dimensional sensor 204 (block 504). The threshold distance(s)correspond to, for example, the first and second threshold distances 114and 118 of FIG. 1. For those of the data points having a depth value,for example, between the threshold distances 114 and 118, the examplethree-dimensional data analyzer 300 performs one or more 3D-basedrecognition analyses to detect people present in the environment 100(block 506). For example, the example body recognizer 308 of FIG. 3executes a plurality of 3D-based recognition analyses to detectdifferent body parts of a person, such as a head, an arm, a leg, etc.Additionally or alternatively, the example skeletal framework generator310 of FIG. 3 generates a mapping of a detected body. Additionally oralternatively, the example face recognizer 312 of FIG. 3 identifies anyknown people present in the environment 100. Additionally oralternatively, the example gesture recognizer 314 of FIG. 3 determineswhether the detected people are making one or more known gestures (e.g.,in connection with the video game system 108 and/or as feedback to themeter 106 in response to media currently being presented by theinformation presentation device 102). As described above, the datapoints identified within the first threshold distance 114 from thesensor 104 may additionally or alternatively be subject to 2D-basedrecognition analysis.

If the example body recognizer 308 of FIG. 3 determines that one or moreobjects in the environment 100 correspond to people (block 508), therelated information is stored in the example body tracker 304 of FIG. 3(block 510). For example, the body recognizer 308 of FIG. 3 conveys anindication of a detected person to the example body counter 316. Theexample body counter 316 maintains records of body counts for differentframes and/or periods of time such that people counts can be generated(e.g., for purposes of generating ratings information for media detectedin the environment 100). In the illustrated example, the body recognizer308 also records the location information (e.g., an x-y coordinate) ofany detected people and/or conveys the location information to theexample crossover detector 320.

The example analysis trigger 326 of FIG. 3 determines whether a 2D-basedrecognition analysis is to be triggered (block 512). FIG. 6 illustratesan example implementation of block 512 of FIG. 5. In the example of FIG.6, the analysis trigger 326 determines whether a predefined interval orperiod of time has elapsed according to, for example, a userconfigurable schedule (block 600). If the predefined interval haselapsed in the example of FIG. 6 (block 600), the example analysistrigger 326 initiates the face recognizer 324 to perform a full scan ofthe environment 100 or a scan of portions of the environment 100 havinga depth value (e.g., according to the corresponding three-dimensionaldata points) greater than the threshold distance (e.g., the secondthreshold distance 118 of FIG. 1) (block 602).

Alternatively, when the predefined interval has not elapsed (block 600),the example analysis trigger 326 determines whether a person hastravelled from within the threshold distance to outside the thresholddistance (block 604). In the illustrated example, the analysis trigger326 references the crossover detector 320 of FIG. 3, which may convey anindication of such a detection to the analysis trigger 326, to determinewhether such a transition from one of the ranges 112, 116 and 120 ofFIG. 1 to another one of the ranges in a direction away from thethree-dimensional sensor 204. If a person is detected travelling outsidethe threshold distance, the example analysis trigger 326 of FIG. 3triggers a targeted 2D-based recognition analysis on a location of theenvironment 100 corresponding to the detected person (block 606). Forexample, the analysis trigger 326 may cause the face recognizer 324 ofFIG. 3 to inspect a rectangle defined around a coordinate of thedetected person to obtain an identity of the person.

In the illustrated example of FIG. 6, the analysis trigger 326 alsodetermines whether one or more previous 2D-based recognition analysis(e.g., performed by the example body recognizer 322) of one or moreimmediately previous frames (e.g., one or more frames directly prior toa current frame in a chronological sequence) resulted in a detection ofa person in the environment 100 (block 608). The number of previousframes that a person detection can trigger a subsequent 2D-basedrecognition analysis is configurable and can be any suitable number offrames (e.g., five frames, ten frames, etc.) and/or a correspondingperiod of time (e.g., ten seconds, fifteen seconds, etc.). As describedabove, such detections are recorded in the example body tracker 304 ofFIG. 3 along with related information, such as a location of thedetected person(s) in the environment 100. In the example of FIG. 6,when a person was detected in a previous frame (block 608), the exampleanalysis trigger 326 of FIG. 3 triggers a targeted 2D-based recognitionanalysis for the location corresponding to the previously detectedperson (block 610).

Alternatively, the example of FIG. 6 may lead to a determination that a2D-based recognition analysis has not been triggered (block 612).Control then returns to FIG. 5. If a 2D-based recognition analysis hasnot been triggered (block 514), control returns to block 502. Otherwise,if a 2D-based recognition analysis has been triggered (block 514), theexample analysis trigger 326 determines what type of 2D-basedrecognition analysis has been triggered (block 516). When the triggered2D-based recognition is a targeted analysis (block 516), the exampleanalysis limiter 328 of FIG. 3 obtains the corresponding location in theenvironment 100 around which the analysis is to be focused (block 518).Further, the example analysis limiter 328 of FIG. 3 defines boundariesto limit the analysis to a certain portion or subset of thetwo-dimensional data captured by the two-dimensional sensor 206 (block518). The focused 2D-based recognition analysis is then executed by, forexample, the body recognizer 322 and/or the face recognizer 324 withinthe defined limit boundaries (block 520). Results of the recognitionanalysis are stored in the example body tracker 304 of FIG. 3 (block522).

When the type of 2D-based recognition analysis that was triggered is afull scan of the environment 100 (block 516), the exampletwo-dimensional data analyzer 302 executes the corresponding one or more2D-based analyses (e.g., via the face recognizer 324 and/or the bodyrecognizer 322) (block 524). The results of the analys(es) are stored inthe example body tracker 304 of FIG. 3 (block 522). Control then returnsto block 502.

FIG. 7 is a block diagram of an example processor platform 700 capableof executing the instructions of FIGS. 5 and 6 to implement the exampleaudience measurement device of FIGS. 1, 2 and/or 3. The processorplatform 700 can be, for example, a personal computer, an Internetappliance, a DVD player, a CD player, a digital video recorder, aBlu-ray player, a gaming console, a personal video recorder, a set topbox, or any other type of computing device.

The processor platform 700 of the instant example includes a processor712. For example, the processor 712 can be implemented by one or moremicroprocessors or controllers from any desired family or manufacturer.

The processor 712 includes a local memory 713 (e.g., a cache) and is incommunication with a main memory including a volatile memory 714 and anon-volatile memory 716 via a bus 718. The volatile memory 714 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM)and/or any other type of random access memory device. The non-volatilememory 716 may be implemented by flash memory and/or any other desiredtype of memory device. Access to the main memory 714 and thenon-volatile memory 716 is controlled by a memory controller.

The example processor platform 700 of FIG. 7 also includes an interfacecircuit 720. The interface circuit 720 may be implemented by any type ofinterface standard, such as an Ethernet interface, a universal serialbus (USB), and/or a PCI express interface.

In the example of FIG. 7, one or more input devices 722 are connected tothe interface circuit 720. The input device(s) 722 permit a user toenter data and commands into the processor 712. The input device(s) canbe implemented by, for example, a keyboard, a mouse, a touchscreen, atrack-pad, a trackball, isopoint and/or a voice recognition system.

In the example of FIG. 7, one or more output devices 724 are alsoconnected to the interface circuit 720. The output devices 724 can beimplemented, for example, by display devices (e.g., a liquid crystaldisplay, a cathode ray tube display (CRT), a printer and/or speakers).The interface circuit 720, thus, typically includes a graphics drivercard.

The interface circuit 720 also includes a communication device such as amodem or network interface card to facilitate exchange of data withexternal computers via a network 726 (e.g., an Ethernet connection, adigital subscriber line (DSL), a telephone line, coaxial cable, acellular telephone system, etc.).

The processor platform 700 also includes one or more mass storagedevices 728 for storing software and data. Examples of such mass storagedevices 728 include floppy disk drives, hard drive disks, compact diskdrives and digital versatile disk (DVD) drives. The mass storage device728 may implement the frame database 318 of FIG. 3.

Coded instructions 732 of FIGS. 5 and/or 6 may be stored in the massstorage device 728, in the volatile memory 714, in the non-volatilememory 716, and/or on a removable storage medium such as a CD or DVD.

In some example implementations, the example meter 106 of FIGS. 1, 2and/or 3 is implemented in connection with an XBOX® gaming system. Insome examples, the one or more sensors associated with the example imagecapturing device 104 of FIGS. 1 and/or 2A are implemented with KINECT®sensors (e.g., to capture images of an environment to count people). Insome examples, some or all of the machine readable instructions of FIGS.5 and/or 6 can be downloaded (e.g., via the Internet) to and stored onan XBOX® gaming console that implements the example meter 106 of FIGS.1, 2 and/or 3. In some examples, the machine readable instructions ofFIGS. 5 and/or 6 are downloaded prior to the retail point of sale andonly activated upon receiving consent from a purchaser.

Although certain example apparatus, methods, and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all apparatus,methods, and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. A method, comprising: analyzing, with aprocessor, a plurality of three-dimensional data points havingrespective depth values representative of distances between a sensor andrespective objects of an environment; when a first set of thethree-dimensional data points has a first depth value less than athreshold, executing, with the processor, a first type of recognitionanalysis on a first area of the environment corresponding to the firstset of the three-dimensional data points; and when a second set of thethree-dimensional data points has a second depth value greater than thethreshold, executing, with the processor, a second type of recognitionanalysis different than the first type of recognition analysis on asecond area of the environment corresponding to the second set of thethree-dimensional data points.
 2. A method as defined in claim 1,wherein the first type of recognition analysis analyzes thethree-dimensional data points in the first set, and the second type ofrecognition analysis analyzes a third set of two-dimensional datarepresentative of at least a portion of the environment.
 3. A method asdefined in claim 1, wherein the three-dimensional data points arecaptured via a depth sensor using electromagnetic radiation beamsprojected in a pattern, and the two-dimensional data is captured via acamera.
 4. A method as defined in claim 1, further comprising storing atwo-dimensional coordinate of the first set in association with a firstframe.
 5. A method as defined in claim 4, further comprising using thetwo-dimensional coordinate to focus the second type of recognitionanalysis on the first area of the environment for a second framesubsequent to the first frame.
 6. A method as defined in claim 1,further comprising triggering execution of the second recognitionanalysis in response to elapsing of a time interval.
 7. A method asdefined in claim 1, wherein the first and second types of recognitionanalyses determine whether the respective objects of the environmentcorrespond to people, and further comprising calculating a people countusing results of the at least one of the first or second types ofrecognition analyses.
 8. A tangible machine readable storage mediumcomprising instructions that, when executed, cause a machine to atleast: execute, in connection with a first frame of data, athree-dimensional recognition analysis on an object detected in anenvironment within a threshold distance from a sensor; store a locationof the object; and in response to detecting that the first object hasmoved outside the threshold distance from the sensor in connection witha second frame subsequent to the first frame: execute a two-dimensionalrecognition analysis on the object in the second frame; and use thestored location of the object as a focal point for the two-dimensionalanalysis.
 9. A storage medium as defined in claim 8, wherein theinstructions cause the machine to use the stored location of the objectas the focal point for the two-dimensional recognition analysis bylimiting the two-dimensional recognition analysis to a portion of thesecond frame associated with the stored location of the object in thefirst frame.
 10. A storage medium as defined in claim 8, wherein theinstructions cause the machine to use the stored location of the objectas the focal point for the two-dimensional recognition analysis bylimiting the two-dimensional recognition analysis to a portion of thesecond frame corresponding to a shape and surrounding the storedlocation of the object in the frame.
 11. A storage medium as defined inclaim 8, wherein the instructions cause the machine to store thelocation of the object by recording a two-dimensional coordinate inassociation with the first frame.
 12. A storage medium as defined inclaim 8, wherein the instructions cause the machine to analyzethree-dimensional data captured of the environment to determine whetherthe object is within the threshold distance from the sensor.
 13. Astorage medium as defined in claim 8, wherein the instructions cause themachine to execute the two-dimensional recognition analysis on theenvironment in response to occurrence of a time event.
 14. A storagemedium as defined in claim 8, wherein the three-dimensional and thetwo-dimensional recognition analyses determine whether the respectiveobjects of the environment correspond to people.
 15. A storage medium asdefined in claim 14, wherein the instructions cause the machine tocalculate a people count using results of at least one of thethree-dimensional or the two-dimensional recognition analyses.
 16. Anaudience measurement device, comprising: a first data analyzer toexecute a first recognition analysis on three-dimensional datarepresentative of a first object within a threshold distance from the athree-dimensional sensor; a second data analyzer to execute a secondrecognition analysis on two-dimensional data representative of a secondobject outside the threshold distance from the three-dimensional sensor;and a counter to combine a first detection of a first person provided bythe first data analyzer and a second detection of a second personprovided by the second data analyzer to generate a people count for anenvironment.
 17. An apparatus as defined in claim 16, further comprisinga crossover detector to detect the first person travelling from withinthe threshold distance to outside the threshold distance, the crossoverdetector to record a location of the first person when the first persontravelled outside the threshold distance.
 18. An apparatus as defined inclaim 17, further comprising a limiter to focus the second recognitionanalysis on the location of the first person.
 19. An apparatus asdefined in claim 18, wherein the limiter is to define a boundary withinthe two-dimensional data in which to perform the second recognitionanalysis to areas adjacent to the location of the first person.
 20. Anapparatus as defined in claim 19, wherein a shape of the boundary to bedefined by the limiter depends on a type of body part detected by thefirst data analyzer.